LoBSTr: Real?time Lower?body Pose Prediction from Sparse Upper?body Tracking Signals

With the popularization of game and VR/AR devices, there is a growing need for capturing human motion with sparse set tracking data. In this paper, we introduce deep neural-network (DNN) based method real-time prediction lower-body pose only from signals upper-body joints. Specifically, our Gated Recurrent Unit (GRU)-based recurrent architecture predicts feet contact probability past sequence head, hands pelvis. A major feature that input signal represented velocity signals. We show representation better models correlation between motions increase robustness against diverse scales proportions user body than position-orientation representations. addition, to remove foot-skating floating artifacts, network state, which used post-process inverse kinematics preserve contact. Our lightweight so as run in applications. effectiveness through several quantitative evaluations other architectures representations, respect wild data obtained commercial VR devices.